IN THE CLAIMS : 

Please cancel Claims 2, 6, 7, 9-1 1 and 14, without prejudice to or disclaimer 
of the subject matter recited therein. 

Please amend Claims 1, 3-5, 8, 12, 13, 15-27, as follows: 



1 . (Currently Amended) An information presentation apparatus 

comprising: 

user operation input unit, adapted to input an operation of a user; 

user viewpoint position and orientation pose measurement unit, adapted to 
measure a position and orientation of pose-at a user's viewpoint; 

input unit, adapted to input viewpoint position and orientation information 
of an other user; 

model data storage unit, adapted to store virtual world model datarreai 
world model data, and data necessary to generate a virtual world image ; 

annotation data storage unit, adapted to store annotation data necessary to be 
added to a real world and a virtual world and then displayed ; 

virtual image generation unit, adapted to generate a virtual world image by 
using measured results of the position and the orientation of the user's viewpoint an image 
of the virtual world by using information in said user viewpoint position and pose 
m e asurem e nt unit, said model data storage unit and said annotation data storage unit ; 

annotation image generation unit, adapted to generate an annotation image 
from the annotation data, based on position and orientation information of the user and the 
viewpoint position and orientation information of the other user; 
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user viewpoint image input unit, adapted to capture an image of the real 
world viewed from the user's viewpoint; and 

composite unit, adapted to composite the image of the real world, the virtual 
world image and the annotation image; and 

image display unit, adapted to display a composite image acquired by said 
composite unit an image obtained by synthesizing the image generated by said virtual 
image gen e ration unit and the image obtained by said us e r viewpoint image input unit, on 
an image display device of the user . 

Claim 2 (Cancelled). 

3. (Currently Amended) An information presentation apparatus 
according to Claim L Claim 2, wherein said input unit inputs, in addition to the viewpoint 
position and orientation information of the other user, identification information of the 
other user and operation information of the other user the communication data includes an 
identification number of each user using said information p r esentation apparatus, a name 
for discriminating each user, position and pose information of each user's viewpoint, 

operation information of each user, and annotation data . 

4. (Currently Amended) An information presentation apparatus 
according to Claim 1, wherein the virtual world model data includes three-dimensional 
coordinates of vertices of a polygon of a virtual computer graphics (CG) object arranged in 



on the virtual world, structure information of faces of the polygon, discrimination 
information of the CG object, color information, texture information, a size of the CG 
object, and position and pose information indicating the arrangement of the CG object on 
the virtual world. 

5. (Currently Amended) An information presentation apparatus 
according to Claim 1, wherein 

said model data storage unit stores real world model data, and 
the real world model data includes three-dimensional coordinates of vertices 
of a polygon of an object existing in the real world merged with the virtual world, structure 
information of faces of the polygon, discrimination information of the object, a size of the 
object, and position and pose information indicating the arrangement of the object. 

Claims 6 and 7 (Cancelled). 

8. (Currently Amended) An information presentation apparatus 
according to Claim L Claim 7, wherein the annotation data includes position and pose 
information of an object arranged on the real world and the virtual wo r ld , discrimination 
information of the object, and text, symbol and image information for indicating 
information of the object to the user . 



Claims 9-11 (Cancelled). 



12. (Currently Amended) An information presentation apparatus 
according to Claim L Claim 11, wherein the annotation is text, symbols or images includes 
a symbol, a character string, and image information . 

13. (Currently Amended) An information presentation apparatus 
according to Claim 1, Claim 9, wherein said annotation virtual image generation unit 
recognizes a target object according to the operation of a user, and generates an annotation 
concerning the recognized target object has a function to automatically recognize a ta r get 
that th e user pays attention . 



Claim 14 (Cancelled). 



1 5 . (Currently Amended) An information presentation apparatus 
according to Claim L Claim 9, wherein said annotation virtual image generation unit 
generates, when a target object of the other user is outside a visual range of the user, an 
annotation indicating a direction of the target has a function, in a case where a target that 
other one or mo r e users pay attention is outsid e a visual range of the user, to generate an 
annotation indicating a direction of the ta r get . 



16. (Currently Amended) An information presentation apparatus 
according to Claim 1, Claim 9, wherein said virtual annotation image generation unit 
generates an annotationfor discriminating a target object of the other user 3 
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a case where a target that othe r one or more us e rs pay attention is inside a visual range of 
the user, to generate an annotation indicating information of the target . 

17. (Currently Amended) An information presentation apparatus 
according to Claim L Claim 9, wherein said annotation virtual image generation unit 
generates an annotation for indicating the other user has a function to generate an 
annotation of which the attributes of a color, a shape and a characte r type have been 
changed in regard to each user, and an annotation indicating a name for discriminating the 
user. 

1 8. (Currently Amended) An information presentation apparatus 
according to Claim 1. Claim 9, wherein said annotation virtual image generation unit has a 
function capable of controlling a generated annotation, by the user's operation input to said 
user operation input unit. 

19. (Currently Amended) An information presentation apparatus 

comprising: 

user operation input unit, adapted to input an operation of a user; 

user viewpoint position and orientation measurement unit, adapted to 
measure a position and orientation of a user's viewpoint; 

model data storage unit, adapted to store virtual world model data, real 
world model data, and data necessary to generate a virtual world image; 
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annotation data storage unit, adapted to store data necessary to be added to a 
real world and a virtual world and then displayed; 

virtual image generation unit, adapted to generate an image of the virtual 
world by using information in said user viewpoint position and orientation measurement 
unit, said model data storage unit and said annotation data storage unit; 

user viewpoint image input unit, adapted to capture an image of the real 
world viewed from the user's viewpoint; and 

image display unit, adapted to display an image obtained by synthesizing the 
image generated by said virtual image generation unit and the image obtained by said user 
viewpoint image input unit, on an image display device of the user, 

wherein said virtual image generation unit draws the information stored in 
said model data storage unit from the user's viewpoint in computer graphics to generate the 
image of the virtual world viewed from the user's viewpoint, by using the position and 
orientation information at the user's viewpoint obtained from said user viewpoint position 
and orientation measurement unit, and according to Claim 9, wherein said virtual image 
generation unit has a function to generate an annotation indicating information of a target 
that the user pays attention to, in a state that its attributes of a color, a shape and a character 
type have been different from those of other annotation. 

20. (Currently Amended) An information presentation apparatus 
according to Claim 1, Claim 9, wherein said annotation virtual image generation unit 
generates, when the other user is outside a visual range of the user, an annotation indicating 
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a direction of the other user has a function to generat e an annotation indicating a direction 
of other user e xisting outside a visual r ange of the user . 



21 . (Currently Amended) An information presentation apparatus 
according to Claim 1, Claim 9, wherein said annotation virtual image generation unit 
generates an annotation for discriminating the other user has a function to generate an 
annotation indicating a position of other user existing inside a visual range of the user . 

22. (Currently Amended) An information processing method 
comprising the steps of: 

inputting viewpoint information of a user; 

generating a virtual world image according to the viewpoint information, by 
using previously held virtual world data; 

inputting viewpoint information of an other user; 

generating an annotation concerning an attention target based on the 
viewpoint information of the user and the viewpoint information of the other user ; and 

generating an image obtained by synthesizing an image of a real world, 
generated virtual world image and the generated annotation. 

23. (Currently Amended) An information processing method according 
to Claim 22, wherein said annotation generation step is adapted to generate, when a target 
object of the other user is outside a visual range of the user, an annotation indicating a 
direction of the target , in a cas e whe r e the attention target exists outside the synthesized 
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image, an annotation indicating a direction of the attention target is generated and 
synth e sized to the synthesized image . 

24. (Currently Amended) An information processing method according 
to Claim 22, wherein said annotation generation step is adapted to generate an annotation 
for discriminating a target object of the other use r , in a case where the attention target 
exists inside the synth e sized imag e , an annotation indicating additional information fo r the 
attention target and having an attribute different from that of othe r annotation is generated 
and synthesized to the synthesized image . 

25. (Currently Amended) An information processing method 
comprising the steps of: 

inputting viewpoint information of a user; 

generating a virtual world image according to the viewpoint information, by 
using previously held virtual world data; 

generating an annotation concerning an attention target; and 

generating an image obtained by synthesizing an image of a real world, 
generated virtual world image and the generated annotation according to Claim 22 , 

wherein an annotation indicating whether or not the attention target is being 
observed by other user is generated and merged synthesized to the synthesized image. 
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26. (Currently Amended) An information processing method according 
to Claim 22, wherein an annotation indicating a position of other user is generated and 
merged synthesized to the synthesized image. 

27. (Currently Amended) A computer-executable program for causing a 
computer to achieve an information processing method comprising the steps of: 

inputting viewpoint information of a user; 

generating a virtual world image according to the viewpoint information, by 
using previously held virtual world data; 

inputting view point information of an other user; 

generating an annotation concerning an attention target based on the 
viewpoint information of the user and the viewpoint information of the other user ; and 

generating an image obtained by synthesizing an image of a real world, 
generated virtual world image and the generated annotation. 



- 11 - 



